The Role of Community Feedback in the Example Annotating Process: an Evaluation on AnnotEx

Example based learning in programming language is a common approach when mastering the art of programming. It encourages students to reuse the code of previously analyzed examples in solving a new problem [1][2]. Gomez-Albarran [3] in a synthesis report about teaching and learning of programming stressed that example-based learning is a natural way of learning. To support online learning from examples in programming courses, WebEx (Web Examples) System was developed to provide interactive access to examples enhanced with line-by-line comments [4]. It allows students to browse the comments at their own pace and order [5][Figure 1]. NavEx (Navigation to Examples) was presented in 2004 to provide adaptive navigation support [6]. The problem addressed in this paper is that teachers usually have limited time to annotate the huge amount of examples. Examples are simply so many, but annotations are so few. In light of this problem, we explore the feasibility of an alternative authoring approach-community-based development of annotations. By harnessing students' power to create example annotations, it not only removes the burden from the instructors but also allows teachers to focus on other pedagogical tasks. In order to collect annotations from student community, we designed the system, AnnotEx (Example Annotator), as a community based authoring environment. We also conducted a pilot study to investigate this research issue [7]. The preliminary results show that community is capable to give ratings and indicate good or bad annotations. Annotations were improved after re-annotation. In addition, in the context of a programming course, authoring (rather than only using) examples could be considered as a useful learning activity. Jonassen and Reeves [8] contend that students are likely to learn more by constructing hypermedia instructional materials that by studying hypermedia created by others. Meanwhile, Chi et al. [9] showed that self-explanations in the context of learning about mechanics from worked-out examples had rather dramatic effects on participants' ability to solve problems on their own. Therefore, the problems we would like to look into are how the student community provides feedback and what the impacts on the example annotations are. Do students really learn from the annotating process and peer-review? Hence, in this paper, we expand the pile of studies in two dimensions. Firstly, we design a clear-cut experiment with control and experimental groups and examine the thorough community effects. Secondly, we analyze the results and aim to answer the additional questions regarding to student learning activities. The …